[GLUTEN-6887][VL] Daily Update Velox Version (2026_04_01) by GlutenPerfBot · Pull Request #11860 · apache/gluten

GlutenPerfBot · 2026-04-01T11:17:16Z

Upstream Velox's New Commits:

24e6ab97b by Chengcheng Jin, fix(cudf): Fix complex data type name in format conversion and add tests(Part1) (16818)
d92b90029 by Natasha Sehgal, refactor: Propagate CastRule cost through canCoerce (16821)
361a42252 by Rui Mo, fix(fuzzer): Reduce Spark aggregate fuzzer test pressure (16964)
2c2fe2ab7 by root, fix: Ignore string column statistics for parquet-mr versions before 1.8.2 (16744)
7faf27a86 by Chengcheng Jin, feat(cudf): Add the log to show detailed fallback messgae (16900)
e603315e5 by Chang chen, feat(parquet): Add type widening support for INT and Decimal types with configurable narrowing (16611)
1e1674dd8 by Rajeev Singh, docs: Add blog post for Adaptive per-function CPU tracking (16945)
0c6b89d61 by Masha Basmanova, fix(build): Guard fuzzer examples subdirectory with VELOX_BUILD_TESTING (16992)
8d6355d8d by Pratik Pugalia, build: Improve build impact comment layout (16971)
44d561990 by Masha Basmanova, refactor: Add ConnectorRegistry class with tryGet and unregisterAll (16977)
793f13f16 by Rajeev Singh, feat(expr-eval):Adaptive per-function CPU sampling for Velox expression evaluation (16646)
1a4dc7a5a by Pratik Pugalia, fix: Off-by-one boundary bug in make_timestamp validation (16944)
7f2c75c26 by Pratik Pugalia, Fix incorrect substr length in Tokenizer::matchUnquotedSubscript (16972)
22b90045e by Masha Basmanova, docs: Add truncate markers to blog posts for cleaner listing page (16975)

velox_branch: https://github.com/IBM/velox/commits/dft-2026_04_01

Related issue: #6887

github-actions · 2026-04-01T20:07:01Z

Run Gluten Clickhouse CI on x86

github-actions · 2026-04-01T20:23:19Z

Run Gluten Clickhouse CI on x86

github-actions · 2026-04-01T21:17:06Z

Run Gluten Clickhouse CI on x86

zhouyuan · 2026-04-01T21:34:33Z

...g/apache/spark/sql/execution/datasources/parquet/GlutenParquetThriftCompatibilitySuite.scala

      "/test-data/parquet-thrift-compat.snappy.parquet"

-  testGluten("Read Parquet file generated by parquet-thrift") {
+  // TODO: https://github.com/apache/gluten/issues/11865


@baibaichen seems due to missing fix from one old OAP patch: https://github.com/IBM/velox/pull/35/changes

it's in baibaichen/velox@9f58f05

waiting for facebookincubator/velox#16966

zhouyuan · 2026-04-02T07:30:00Z

Run Gluten Clickhouse CI on x86

github-actions · 2026-04-02T09:05:45Z

Run Gluten Clickhouse CI on x86

github-actions · 2026-04-02T14:14:10Z

Run Gluten Clickhouse CI on x86

zhouyuan · 2026-04-02T14:15:09Z

...ickhouse/src/test/scala/org/apache/gluten/execution/GlutenClickHouseTPCDSAbstractSuite.scala

  protected val tablesPath: String = UTSystemParameters.tpcdsDecimalDataPath + "/"
  protected val db_name: String = "tpcdsdb"
+  // TODO: fix to use the new DS queries https://github.com/apache/gluten/issues/11871
  protected val tpcdsQueries: String =


zhouyuan · 2026-04-02T16:46:11Z

Run Gluten Clickhouse CI on x86

github-actions · 2026-04-02T16:48:09Z

Run Gluten Clickhouse CI on x86

zhouyuan · 2026-04-02T19:14:59Z

Run Gluten Clickhouse CI on x86

github-actions · 2026-04-02T21:27:10Z

Run Gluten Clickhouse CI on x86

github-actions · 2026-04-03T06:28:14Z

Run Gluten Clickhouse CI on x86

github-actions · 2026-04-03T08:42:53Z

Run Gluten Clickhouse CI on x86

Upstream Velox's New Commits: 24e6ab97b by Chengcheng Jin, fix(cudf): Fix complex data type name in format conversion and add tests(Part1) (#16818) d92b90029 by Natasha Sehgal, refactor: Propagate CastRule cost through canCoerce (#16821) 361a42252 by Rui Mo, fix(fuzzer): Reduce Spark aggregate fuzzer test pressure (#16964) 2c2fe2ab7 by root, fix: Ignore string column statistics for parquet-mr versions before 1.8.2 (#16744) 7faf27a86 by Chengcheng Jin, feat(cudf): Add the log to show detailed fallback messgae (#16900) e603315e5 by Chang chen, feat(parquet): Add type widening support for INT and Decimal types with configurable narrowing (#16611) 1e1674dd8 by Rajeev Singh, docs: Add blog post for Adaptive per-function CPU tracking (#16945) 0c6b89d61 by Masha Basmanova, fix(build): Guard fuzzer examples subdirectory with VELOX_BUILD_TESTING (#16992) 8d6355d8d by Pratik Pugalia, build: Improve build impact comment layout (#16971) 44d561990 by Masha Basmanova, refactor: Add ConnectorRegistry class with tryGet and unregisterAll (#16977) 793f13f16 by Rajeev Singh, feat(expr-eval):Adaptive per-function CPU sampling for Velox expression evaluation (#16646) 1a4dc7a5a by Pratik Pugalia, fix: Off-by-one boundary bug in make_timestamp validation (#16944) 7f2c75c26 by Pratik Pugalia, Fix incorrect substr length in Tokenizer::matchUnquotedSubscript (#16972) 22b90045e by Masha Basmanova, docs: Add truncate markers to blog posts for cleaner listing page (#16975) Signed-off-by: glutenperfbot <glutenperfbot@glutenproject-internal.com>

…olumns When Gluten creates HiveTableHandle, it was passing all columns (including partition columns) as dataColumns. This caused Velox's convertType() to validate partition column types against the Parquet file's physical types, failing when they differ (e.g., LongType in file vs IntegerType from partition inference). Fix: build dataColumns excluding partition columns (ColumnType::kPartitionKey). Partition column values come from the partition path, not from the file. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

With OAP INT narrowing commit replaced by upstream Velox PR #15173: - Remove 2 excludes now passing: LongType->IntegerType, LongType->DateType - Add 2 excludes for new failures: IntegerType->ShortType (OAP removed) Exclude 63 (net unchanged: -2 +2). Test results: 21 pass / 63 ignored. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

This suite tests the READ path only. Disable native writer so Spark's writer produces correct V2 encodings (DELTA_BINARY_PACKED/DELTA_BYTE_ARRAY). - Remove 10 excludes for decimal widening tests now passing Remaining 38 excludes: - 34: Velox native reader rejects incompatible decimal conversions regardless of reader config (no parquet-mr fallback) - 4: Velox does not support DELTA_BYTE_ARRAY encoding Test results: 46 pass / 38 ignored. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

Velox native reader always behaves like Spark's vectorized reader, so tests that rely on parquet-mr behavior (vectorized=false) fail. Instead of just excluding these 33 tests, add testGluten overrides with expectError=true to verify Velox correctly rejects incompatible conversions. - 16 unsupported INT->Decimal conversions - 6 decimal precision narrowing cases - 11 decimal precision+scale narrowing/mixed cases VeloxTestSettings: 38 excludes (parent tests) + 33 testGluten overrides Test results: 79 pass / 38 ignored (33 excluded parent + 5 truly excluded)

Signed-off-by: Yuan <yuanzhou@apache.org>

the testing data on clickhouse side is not upated, so revert to use the old query Signed-off-by: Yuan <yuanzhou@apache.org>

Signed-off-by: Yuan <yuanzhou@apache.org>

github-actions · 2026-04-03T11:06:46Z

Run Gluten Clickhouse CI on x86

zhouyuan · 2026-04-03T11:08:44Z

backends-clickhouse/src/test/scala/org/apache/gluten/execution/GlutenEliminateJoinSuite.scala


-  test("Eliminate two aggregate joins with attribute reordered") {
+  ignore("Eliminate two aggregate joins with attribute reordered") {
    val sql = """


@zzcclp this test failed, it's not related with this patch, seems due to the recent changes in the past two weeks

I will take a look next week.

github-actions bot added BUILD VELOX labels Apr 1, 2026

zhouyuan mentioned this pull request Apr 1, 2026

[GLUTEN-11683][VL] Add Parquet type widening support #11719

Draft

github-actions bot added the CORE works for Gluten Core label Apr 1, 2026

zhouyuan force-pushed the tagging-2026_04_01 branch from f02193f to 82d1a35 Compare April 1, 2026 21:16

zhouyuan reviewed Apr 1, 2026

View reviewed changes

zhouyuan approved these changes Apr 2, 2026

View reviewed changes

This was referenced Apr 2, 2026

[GLUTEN-6887][VL] Daily Update Velox Version (2026_03_28) #11845

Closed

[GLUTEN-6887][VL] Daily Update Velox Version (2026_03_31) #11856

Closed

zhouyuan mentioned this pull request Apr 2, 2026

[VL] Read Parquet file generated by parquet-thrift failed #11865

Open

github-actions bot added TOOLS CLICKHOUSE labels Apr 2, 2026

zhouyuan reviewed Apr 2, 2026

View reviewed changes

zhouyuan force-pushed the tagging-2026_04_01 branch from 07d2ba4 to ef679c9 Compare April 3, 2026 06:27

zhouyuan force-pushed the tagging-2026_04_01 branch from ef679c9 to 3ab0761 Compare April 3, 2026 08:42

glutenperfbot and others added 3 commits April 3, 2026 10:59

Point Velox to PR3 branch with parquet type widening support

4c29b7e

baibaichen and others added 12 commits April 3, 2026 10:59

fix velox rebase

48dd3d8

Signed-off-by: Yuan <yuanzhou@apache.org>

ignore ut

ac91ab2

Signed-off-by: Yuan <yuanzhou@apache.org>

ignore more ut

75e6871

Signed-off-by: Yuan <yuanzhou@apache.org>

fix ignore api

e7f9ba4

Signed-off-by: Yuan <yuanzhou@apache.org>

ignore failed ut

4d99fd3

Signed-off-by: Yuan <yuanzhou@apache.org>

fix on clickhouse tpcds queries

df9b816

the testing data on clickhouse side is not upated, so revert to use the old query Signed-off-by: Yuan <yuanzhou@apache.org>

fix q30

27e83cf

ignore ut

7384ed2

fix

e4499a6

Signed-off-by: Yuan <yuanzhou@apache.org>

zhouyuan force-pushed the tagging-2026_04_01 branch from 3ab0761 to e4499a6 Compare April 3, 2026 11:06

github-actions bot added the INFRA label Apr 3, 2026

zhouyuan reviewed Apr 3, 2026

View reviewed changes

zhouyuan merged commit 7b638f0 into apache:main Apr 3, 2026
114 of 117 checks passed

This comment was marked as off-topic.

Sign in to view

zhouyuan mentioned this pull request Apr 3, 2026

[GLUTEN-1433] [VL] Add config to disable TimestampNTZ validation fallback #11720

Open

Conversation

GlutenPerfBot commented Apr 1, 2026 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Apr 1, 2026

Uh oh!

github-actions bot commented Apr 1, 2026

Uh oh!

github-actions bot commented Apr 1, 2026

Uh oh!

zhouyuan Apr 1, 2026

Choose a reason for hiding this comment

Uh oh!

baibaichen Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

zhouyuan Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

zhouyuan commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

zhouyuan Apr 2, 2026

Choose a reason for hiding this comment

Uh oh!

zhouyuan commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

zhouyuan commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 2, 2026

Uh oh!

github-actions bot commented Apr 3, 2026

Uh oh!

github-actions bot commented Apr 3, 2026

Uh oh!

github-actions bot commented Apr 3, 2026

Uh oh!

zhouyuan Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

zzcclp Apr 3, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This comment was marked as off-topic.

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

GlutenPerfBot commented Apr 1, 2026 •

edited by github-actions bot

Loading